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COMPENSATION FOR OVERFLIGHT VELOCITY WHEN STABILIZING AN 

AIRBORNE CAMERA 

CROSS-REFERENCE TO RELATED APPLICATION 

[0001] The present application claims the benefit of pending U.S. Provisional 

Application No. 60/440,983, filed January 17, 2003 which is incorporated herein in 
its entirety by reference. 

TECHNICAL FIELD 

[0002] The described technology relates to an apparatus for stabilization of the 

pointing direction of an airborne camera. 

BACKGROUND 

[0003] The pointing direction or line of sight of an airborne camera that is fixed to 

the body of an aircraft moves about as the aircraft maneuvers. For example, as 
the aircraft rolls, the camera rolls up or down depending on which side of the 
aircraft it is mounted. If the camera is inertially stabilized, rather than fixed to the 
body of the aircraft, its line of sight will remain in a fixed direction relative to the 
body of the aircraft as the aircraft maneuvers. When the camera is inertially 
stabilized, its field of view sweeps across a scene at a velocity equal to the 
overflight velocity of the aircraft. For example, if the camera is pointing directly 
down with its scan aligned with the body of the aircraft and the aircraft is traveling 
at 360 mph and if the camera's field of view is one mile, then an object entering 
the field of view will exit the field of view in 10 seconds. 

[0004] Inertial stabilization is typically effected by mounting an airborne camera 

within a gimbal system, and driving the gimbals based on the roll, pitch, and 
heading of the aircraft. Such an approach can deliver high-quality stabilized 
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pointing, but aircraft overflight velocity is not removed. Operator inputs are used 
to command changes in line of sight to compensate for overflight velocity. Such 
systems are the subject of U.S. Patent 5,897,223; U.S. Patent 3,638,502; U.S. 
Patent 4,989,466; U.S. Patent 4,643,539; and U.S. Patent 5,184,521. Figure 1 
illustrates a prior technique that does not correct for overflight velocity of the 
aircraft over the target. In such prior art, the line of sight of the camera is inertially 
stabilized. In this example, as the aircraft 100 circles 101, the camera's line of 
sight sweeps out a circle 102. This stabilization sweeps the imaged area across 
the ground at a velocity equal to aircraft overflight velocity. To compensate for 
the overflight velocity, an operator would manually need to keep the camera 
pointing at the target. 

[0005] Automatic compensation for aircraft overflight velocity causes the line of 

sight to remain fixed upon a selected position relative to the ground. If the gimbal 
system performs this compensation automatically, then the operator is relieved of 
this burden. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0006] Figure 1 illustrates a prior technique requiring operator correction for 

overflight velocity of the aircraft over the target. 
[0007] Figure 2 illustrates the compensation system in which the line of sight of 

the camera is not inertially stabilized but is rather given an angular velocity. 
[0008] Figure 3 illustrates the effect of this compensation upon the image 

sequence created by the camera. 
[0009] Figure 4 illustrates how the required angular velocity of the line of sight is 

calculated. 

[0010] Figure 5 is a diagram illustrating the reference frames used in the 

compensation system. 

[0011] Figure 6 is a flow diagram illustrating the processing of a component that 

establishes the position of the target in one embodiment. 
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[0012] Figure 7 is a flow diagram illustrating the processing of a component that 

locks on the target in one embodiment. 

DETAILED DESCRIPTION 

[0013] A method and system for maintaining the line of sight of an airborne 

camera fixed on a target by compensating for overflight velocity of the aircraft is 
provided. The compensation system automatically commands an angular velocity 
of the line of sight to maintain the camera pointing at the target (e.g., a selected 
location on the ground) being overflown. This angular velocity of the line of sight 
is computed based upon the aircraft overflight velocity and upon a vector from the 
aircraft to the target. This automatic compensation for aircraft overflight velocity 
causes the line of sight to remain fixed upon the target. The compensation 
system drives a gimbal system upon which the camera is mounted to perform this 
compensation automatically. As a result, the camera can remain pointed at the 
target without needing operator intervention. 

[0014] In one embodiment, the compensation system calculates the radians per 

second in the scan and tilt directions that the camera needs to move to 
compensate for the overflight velocity of the aircraft. The compensation system 
periodically (e.g., 20 times per second) calculates the radians and adjusts the 
gyro's angular control of the gimbal system. The compensation system calculates 
both a static and a dynamic adjustment. The static adjustment represents the 
difference in radians between the actual line of sight of the camera and the 
needed line of sight of the camera based on the current positions of the aircraft 
and the target. The dynamic adjustment represents the radians per second that 
the camera needs to move to compensate for the overflight velocity. The 
compensation system combines the static and dynamic adjustments to give the 
actual adjustment. The dynamic adjustments allow for smooth and continuous 
adjustments to be made based on velocity, whereas the static adjustments tend to 
correct for accumulated errors in the line of sight. The compensation system may 
apply a weighting factor to the static and dynamic adjustments. For example, if 
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the position of the aircraft is not very accurate, then the static adjustment may be 
given a low weight relative to the dynamic adjustment because the dynamic 
adjustment would be assumed to be more accurate. By combining the static and 
dynamic adjustments, the compensation system can more accurately keep the line 
of sight of the camera on the target. 

[0015] Figure 2 illustrates the line of sight of a camera that is given an angular 

velocity by the compensation system. This angular velocity compensates for the 
overflight velocity of the aircraft with respect to the target. The center of the 
imaged area is held fixed upon a location on the ground. The aircraft 200 is 
traveling at a velocity represented by vector Vne D . Initially, the line of sight of the 
camera is in the direction represented by vector 201. As the aircraft flies at 
velocity Vned, the angular velocity of the camera is represented by vector 207. 
The line of sight of the camera transitions through vector 201 to vector 206 in 
order to keep the line of sight on the target. 

[0016] Figure 3 illustrates the effect of the compensation upon the image 

sequence created by a camera. This compensation causes the field of view to 
remain fixed upon a selected location, rather than sweep along at aircraft 
overflight velocity. In this example, as the aircraft 300 circles 301, the camera's 
line of sight stays centered on target 302. 

[0017] Figure 4 illustrates calculation of the angular velocity of the camera to keep 

the line of sight centered on the target. The calculation employs the relative 
velocity between the aircraft and the target, together with the vector from the 
aircraft to the target. This calculation is performed continuously, as the aircraft 
velocity and position change and as the target location changes. The 
compensation system initially acquires 401 the target position by, for example, an 
operator centering the camera on the target. The compensation system can 
calculate the position of the target based on the altitude of the target (e.g., sea 
level), aircraft altitude and position, and angles of the gimbal system. The 
compensation system then updates 402 the line of sight based on the current 
aircraft position and the target position. The compensation system calculates 403 
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the rate of change of the line of sight based on the current aircraft position and 
the target position along with relative velocity between aircraft and target. The 
update of the line of sight is the static adjustment, and the update of the rate of 
change is the dynamic adjustment. The compensation system loops calculating 
the static and dynamic adjustments. The dynamic adjustment allows for the 
adjusting of the line of sight continuously, that is the gimbals are moved at the 
specified angular rate between calculations. The static adjustment allows for the 
line of sight to be corrected due to accumulated errors caused by the dynamic 
adjustment and prior static adjustments. If the dynamic adjustment were not 
made, the error between the static adjustments would be much larger. 

[0018] Figure 5 is a diagram illustrating the reference frames used in the 

compensation system in one embodiment. The reference frame of the earth is 
represented by north N, east E, and down D coordinates. The position of an 
aircraft, which may be provided by an altimeter and a GPS system, is in the earth 
reference frame. The reference frame of the body of the aircraft is represented by 
heading B1, pitch B2, and roll B3 coordinates that may be provided by the 
aircraft's gyros. The reference frame of the camera is represented by a line of 
sight C1, tilt C2, and scan C3 coordinates. In one embodiment, the camera is 
controlled by an inertial stabilization system that controls the gimbal motors to 
control the orientation of the C1 , C2, and C3 camera axes. The compensation 
system receives camera scan and tilt rate information from the camera rate gyros 
and adjusts these rates to further account for the overflight velocity. The inertial 
stabilization system maintains the orientation of the camera to compensate for the 
maneuvering of the aircraft. 

[0019] The compensation system inputs the altitude of the target and an indication 

of the line of sight of the camera when pointing at the target. The compensation 
system initially calculates the position of the target in the reference frame of the 
earth R E target using the position of the aircraft R E aircraft. The compensation 
system then repeatedly adjusts the gimbal angles so that the target is in the line 
of sight of the camera to compensate for the overflight velocity. The 
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compensation system calculates the velocity of the aircraft relative to the target 
and adjusts the gimbal angles as the aircraft moves so that the target will remain 
in the line of sight. This adjustment is referred to as the dynamic difference in the 
line of sight because it compensates for movement of the aircraft relative to the 
target. Because the adjustments may not perfectly compensate for the overflight 
velocity, the compensation system also calculates the static difference between 
the actual line of sight of the camera and the line of sight needed to point to the 
target. The compensation system includes this static difference in the adjustment 
to the gimbal angles. Thus, the adjustments for dynamic differences based on 
velocity compensates for the overflight velocity, and the adjustments for static 
differences based on the line of sight of the camera compensates for variations 
between the intended and the actual line of sight. 
[0020] The compensation system uses transformation matrices to represent the 

current orientation of the body of the aircraft relative to the earth reference frame 
and the current orientation of the camera to the body reference frame. The 
camera reference frame relative to the body of the aircraft reference frame is 
represented by a transformation matrix C CB for transforming a vector from the body 
reference frame to the camera reference frame. C CB is a 3-by-3 matrix whose 
columns are orthogonal and normalized, also referred to as a matrix of direction 
cosines. The following equation represents the conversion of a position in the 
body reference frame to the camera reference frame: 

[0021] R C =C CB R B (1) 

[0022] where R B represents the position in the body reference frame and R c represents 
the position in the camera reference frame. An example C CB is 
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[0023] The matrix C CB is set based on the angles of the gimbal relative to the body. 
Thus, this matrix represents the current gimbal angles. A matrix C BE is for 
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transforming from the earth reference frame to the body reference frame. Thus, 
the matrix C BE represents the heading, pitch, and roll of the aircraft as measured 
by the gyro of the aircraft. 
[0024] Figure 6 is a flow diagram illustrating the processing of a component of the 

compensation system that establishes the position of the target in one 
embodiment. The component inputs the position of the aircraft R E jrcraft an 
indication of the pitch, roll, and heading of the aircraft; an indication of the 
direction of the target in the camera reference frame; and the altitude of the 
target. The indication of the pitch, roll, and heading of the aircraft is represented 
by a transformation matrix C BE for transforming a position in the earth reference 
frame to the body reference frame. The indication of the line of sight of the 
camera (which is pointing at the target) is represented by a transformation matrix 
Gcb for transforming a position in the body reference frame to the camera 
reference frame. The direction of the target may be indicated by centering the 
camera on the target. Alternatively, an operator could point to the target as it is 
being displayed using a pointing device such as a mouse. The line of sight of the 
camera is given by vector (1,0,0) in the camera reference frame. Thus, if the 
target is centered in the camera image, then its coordinates are (1,0,0). 
Otherwise, its coordinates have non-zero scan and tilt values. In block 601, the 
component calculates a transformation matrix for transforming from the earth 
reference frame to the camera reference frame as follows: 

[0025] C CE = C CB C BE (3) 

[0026] In block 602, the component calculates the line of sight of the camera in the earth 
reference frame as 

[0027] L E = C CE T (1,0,0) T (4) 

[0028] where L E is the line of sight of the camera in the earth reference frame and where 
the superscript T represents the transform of the matrix or vector. In block 603, 
the component calculates the altitude difference between the aircraft and the 
target. The altitude difference between the aircraft and the target is 
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[0029] AD = D aircraft -D Urget (5) 

[0030] where D represents the down coordinate of the earth reference frame. In blocks 
604-606, the component adjusts L E by AD to convert it to the actual position vector 
AR E from the aircraft to the target. The adjustment multiplies each coordinate by 
AD divided by the down coordinate of the line of sight of the camera in the earth 
reference frame represented as L E (D) as follows: 

arE(d)=lE(d) *i^) 

(0031] AR^N^L^N).^^ (6) 

AR E (E) = L E (E) * ■ AD 



L'(D) 

[0032] where AR E (x) represents the difference in position vector between the aircraft and 
the target in the earth reference frame, L E (x) represents the line of sight of the 
camera in the earth reference frame, x represents a component of the reference 
frame, and AD represents the difference in altitude between the aircraft and the 
target. In an alternate embodiment, the actual position vector from the aircraft to 
the target AR E can be calculated by multiplying the line of sight of the camera L E 
times the range K as follows: 

[0033] AR E = L E * K (7) 

[0034] The range can be determined using a range finder instrument. In block 607, the 
component calculates the position of the target in the earth reference frame as 

[00 35 ] R E rBet =AR E +R E rcraft (8) 

[0036] The component then returns the position of the target Rjj^, . 

[0037] Figure 7 is a flow diagram illustrating the processing of a component that 

locks on the target in one embodiment. The component may be invoked several 
times per second and is passed the current position of the aircraft and target 
(Rfircraft and Rfarget )> an indication of the pitch, roll, and heading of the aircraft (C BE ); 
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and an indication of the line of sight of the camera (C CB ). In block 701, the 
component calculates a transformation matrix for transforming from the earth 
reference frame to the camera reference frame as follows: 

[0038] C CE = Ccb C BE (9) 

[0039] In block 702, the component calculates the position vector ( AR E ) between the 
aircraft and the target in the earth reference frame as: 

I 0040 ! AR E =R^-R a E ircrafl (10) 

[0041] In block 703, the component transforms AR E to the camera reference frame as 
follows: 

[0042] AR C = C CE AR E (11) 

[0043] In block 704, the component normalizes the position vector between the aircraft 
and the target by dividing its coordinates by the distance from the aircraft to the 
target as follows: 

nC AR C 

[0044] AR =-, — - (12) 



AR C 



[0045] where aR c is the normalized position vector and |ar c | is the distance from the 

aircraft to the target. The scan and tilt coordinates of the normalized position 
vector indicate the difference in radians between the actual line of sight of the 
camera and the line of sight needed to point to the target in the scan and tilt 
directions, which represents the static adjustment. In block 705, the component 
calculates the velocity vector (AV E ) between the aircraft and the target in the earth 
reference frame as: 

I 0046 ] AV E =V E get -V E rcraft (13) 

[0047] In block 706, the component transforms the velocity vector AV E to the camera 
reference frame as follows: 

[0048] AV C = C CE AV E (14) 
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[0049] where AV° represents the velocity vector in the camera reference frame. In block 
707, the component normalizes the velocity vector between the aircraft and the 
target by dividing its coordinates by the distance from the aircraft to the target as 

[0050] A^ C =-^- (15) 



[0051] where AV C is the normalized velocity vector and |ar c | is the distance. The scan 

and tilt coordinates indicate the radians per second that the camera needs to 
move to compensate for the overflight velocity in the scan and tilt directions. In 
block 708, the component calculates the adjustment for the signal from the inertial 
stabilizer to the gimbal for the scan of the camera as 

[0052] A(S) = AR C (S)*W+AV C (S) (16) 

[0053] where A(S) is the adjustment for the scan coordinate in radians per second, 
AR C (S) is the scan coordinate of the normalized position vector, AV C (S) is the 

scan coordinate of the normalized velocity vector, and W is a weighting factor for 
combining the velocity and position variation in units of per second. The 
weighting factor can be increased or decreased to compensate for the accuracy of 
the measurements used to calculate the dynamic and static adjustments. For 
example, if the attitude of the aircraft is known to be not very accurate, then the 
weighting factor may be small. Conceptually, AR C (S) is the error between the 
line of sight of the camera and where it should be to point to the target, and 
AV C (S) is the velocity at which the camera should be scanning to keep the target 

in its line of sight. The compensation system compares the adjustment to the 
scan rate of the camera provided by the gyro and uses the difference in rate to 
control the velocity of the gimbal motors. For example, if the gyro indicates that 
the camera should scan at .1 radian per minute and the compensation system 
indicates that the scan rate should be .01 radians per minute, then the rate 
provided to the gimbal system would be .1 1 radians per minute. In block 709, the 
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component calculates the adjustment for the signal from the inertial stabilizer to 
the gimbal for the tilt of the camera in an analogous manner and then returns the 
adjustments. 

[0054] In one embodiment, the compensation system calculates the altitude of the 

target based on a user indicating the position of a target on the camera image. 
For example, a user may center the target within the image at one time and then 
at a later time again center the target within the image. Thus, the compensation 
system has two lines of sight to the target from two different aircraft positions. 
The compensation system can convert these lines of sight to the earth reference 
frame and then calculate their point of intersection in the earth frame of reference, 
which gives the altitude of the target. The compensation system can also 
calculate the velocity of the target based on a series of centering the image on the 
target or a series of indications where the target is within the image (e.g., clicking 
on the target). This velocity and altitude can be provided to compensate for the 
overflight velocity. 

[0055] One skilled in the art will appreciate that although specific embodiments of 

the compensation system have been described for purposes of illustration, 
various modifications can be made without deviating from the spirit and scope of 
the invention. For example, the principles of the compensation system may be 
used on a transport mechanism other than an airplane, such as a satellite, a 
rocket, a missile, a train, an automobile, and so on. In addition, the compensation 
system may be used to control the "line of sight" or "orientation" of a device other 
than a camera, such as a laser or a weapons system. In such a case, a 
positioning scope may be used to establish the initial line of sight to the target. 
Accordingly, the invention is described by the appended claims. 
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